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Abstract 

Electronic portfolios (e- 
portfolios) represent an 
assessment measure with 
strong potential to provide 
feedback about student 
performance to improve 
curricula and pedagogy, 
to determine individual 
students' mastery of learning 
and support feedback 
for improvement, and to 
actively involve students 
in the assessment process. 

This study examined the 
relationship between e- 
portfolio participation and 
student success. Despite 
some limitations, the current 
study demonstrates that, after 
controlling for background 
factors, undergraduate 
students with e-portfolio 
artifacts had significantly 
higher grade-point averages, 
credit hours earned, and 
retention rates than a matched 
set of students without e- 
portfolio artifacts. Also, 
there were significant positive 
relationships between various 
measures of e-portfolio 
utilization and grade-point 
average and credit hours 
earned. There were no 
statistically significant group 
differences in any of the 



National Survey of Student 
Engagement or New Student 
Transition Questionnaire scales, 
which served as measures of 
student academic engagement. 

Background 

Many criticize the state 
of contemporary American 
higher education. Some 
fear that students are not 
developing competencies 
such as communication, critical 
thinking, and a developed 
sense of social responsibility. 
There is increasing skepticism 
concerning the quality 
and utility of a liberal arts 
education. Members of 
the public, employers, and 
legislators are concerned 
with the perceived lack of 
attention that faculty give 
to undergraduate learning 
(American Council on 
Education [ACE], American 
Association of Community 
Colleges [AACC], American 
Association of State 
Colleges and Universities 
[AASCU], Association of 
American Universities, 

[AAU], National Association 
of Independent Colleges 
and Universities [NAICU], 8c 
National Association of State 
Universities and Land Grant 
Colleges [NASULGC], 2006; 
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U. S. Department of Education, 
2006). Colleges and universities 
must respond to these criticisms 
at the same time students are 
arriving on campus with an 
increasingly diverse array of 
experiences, preparation, and 
expectations. 

Several recent longitudinal 
studies across a wide variety 
of institutions have highlighted 
problems affecting the state of 
undergraduate learning in the 
United States. Problems include 
a discontinuity between K-12 
schools and colleges, institutional 
confusion over purposes and 
goals, and the tension between 
the liberal arts and professional 
curricula. Other concerns relate 
to the conflict many faculty 
members experience between 
loyalty to their institutions and 
to their disciplines and between 
their interest in teaching and 
their priorities in research. The 
divisions between academic 
and student affairs on campuses 
create additional challenges. The 
studies highlight the need to 
draw more explicit connections 
between the classes students 
take and their in- and out-of- 
class experiences; to become 
more student-centered; to 
promote student-faculty and 
student-student interaction; 
and to encourage collaborative, 
active learning activities. These 
studies also suggest the need to 
improve student engagement, 
make high expectations explicit, 
and emphasize competency over 
content and collaboration over 
competition (Astin, 1993; Boyer, 
1987; Gamson & Chickering, 
1987; Joint Task Force, 1998; 
Kellogg Commission, 1997; Kuh, 
Schuh, Whitt, & Associates, 1991; 
National Institute of Education, 
1984; Pascarella & Terenzini, 



1991; Schneider & Schoenberg, 
1998).Assessment has been 
suggested by many as a means 
of addressing these problems. 
The "assessment movement" 
that began in the mid-1980s 
has been traced to both an 
extant scholarship of student 
learning and success (e.g., Astin, 
1977; Bowen, 1977; Feldman 
& Newcomb, 1969; Learned & 
Wood, 1938; Pace, 1979; Tinto, 
1975) and especially to a series 
of calls from outside the academy 
to improve accountability (e.g.. 
National Governors' Association, 
1986; National Commission 
on Excellence in Education, 

1983). While 98% of institutions 
reported having an assessment 
program by 1993 (El-Khawas, 
1993), many scholars and 
practitioners have noted that 
assessment has not substantially 
improved student learning at 
most institutions. Ewell (2002) 
notes this lack of success may 
be the result of disagreement 
about the underlying purpose of 
assessment. Is it to benchmark 
institutional performance in the 
name of accountability as in 
K-12 education? Is it intended 
to provide feedback about 
student performance to improve 
curricula and pedagogy? Is its 
goal to determine an individual 
student's mastery of learning 
and to provide feedback for 
improvement? Ewell (2002) 
suggests that for assessment 
to move from its current state 
of "broad but not deep," 
fundamental changes must 
occur. The assessment paradigm 
must shift from "a largely top- 
down, management-oriented" 
evaluation and passive checking 
of results to one of "active and 
collective responsibility for 
fostering student attainment" 



that resides at the level of the 
individual faculty member and 
academic program (p. 24). 

Student portfolios became an 
increasingly popular assessment 
method in the 1990s (Ewell, 
2002). Banta (1999) has 
termed them "the instrument 
of choice for assessment on a 
growing number of campuses" 

(p. 3). Love, McKean, and 
Gathercoal (2004, p. 24) say 
that they "may have the most 
significant effect on education 
since the introduction of formal 
schooling." Portfolios hold 
a high degree of promise for 
accomplishing the last two 
purposes of assessment noted by 
Ewell (2002): providing feedback 
about student performance to 
improve curricula and pedagogy 
as well as determining individual 
students' mastery of learning 
and providing feedback for 
improvement. Additionally, they 
provide students with a planning 
and goal-setting tool that assists 
them in making connections 
between learning experiences, 
provide faculty with a vehicle 
for more authentic discussions 
about teaching and learning, and 
provide institutions with a tool to 
establish a more permanent role 
in the lives of learners (Siemens, 
2004). Also, portfolios achieve a 
goal that many other assessment 
methods can not. They change 
the student role in assessment 
from passive research subject to 
active participant as students are 
called upon to select samples of 
their classroom and co-curricular 
work products or artifacts for 
the portfolio and (perhaps most 
importantly) to reflect upon why 
these artifacts were selected and 
how they demonstrate learning 
(Palomba, 2002). Portfolios are 
not without their challenges as 
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an assessment method. To be 
used successfully, they require a 
great deal of faculty and student 
time and require clear guidelines 
related to purpose, how content 
is evaluated, and how feedback is 
provided. 

In addition to the features 
associated with paper and pencil 
portfolios, electronic (web-based) 
portfolios (e-portfolios) offer 
the advantages of accessibility 
and portability of artifacts, 
faculty/advisor assessments, and 
student reflections. Also, artifact 
formats such as video and sound 
recordings that are difficult to 
include in traditional portfolios 
are easily included in e-portfolios. 
Finally, many e-portfolio software 
packages allow students to 
control who is able to view each 
artifact. They allow reflection 
and assessment, and they permit 
both developmental/assessment 
and showcase (for prospective 
employers, graduate/professional 
schools, etc.) formats to be 
presented (Cambridge, 2001; 
Yancey, 2001). 

Institutional Context 

Bowling Green State University 
(BGSU), a state-assisted, 
residential, doctoral-research- 
intensive university in northwest 
Ohio, has grappled with many of 
the assessment challenges noted 
above. While most academic 
programs have developed 
learning outcomes, created or 
acquired associated measures, 
and collected data — and some 
examples of improvements to 
curricula and pedagogy are 
evident — assessment has not led 
to profound changes in student 
learning or to a widespread 
"culture of evidence." Many 
faculty and nearly all students are 



not aware of assessment efforts 
and a bureaucratic compliance 
mentality still permeates many 
annual assessment reports. At 
the same time, the University has 
articulated as its vision a desire 
to be "the premier learning 
community in Ohio and one 
of the best in the Nation." It 
has developed a wide slate of 
learning communities and other 
student academic enrichment 
programs, identified a set of 
University learning outcomes, 
redesigned its general education 
program, substantially upgraded 
its technology infrastructure, and 
improved its institutional research 
capacity. 

BGSU acquired the Epsilen 
electronic portfolio software in 
2003. As noted above, students 
can place a variety of artifacts 
(e.g., papers, spreadsheets, 
presentations, video and audio 
recordings) and accompanying 
reflections into both a year- 
by-year matrix that shows 
students' progress over time 
for assessment purposes and 
also into a "showcase" version 
of the e-portfolio that might 
be viewed, for example, by 
potential employers or graduate/ 
professional schools. The 
software's reporting capability 
allows the following elements to 
be counted for each participant: 
showcase artifacts (artifacts in the 
showcase version of the student's 
e-portfolio), matrix artifacts 
(artifacts in the matrix version of 
the student's e-portfolio), artifact- 
specific reflections, general 
reflections, total files uploaded to 
the e-portfolio, events posted to 
the student's e-portfolio calendar, 
bookmarks created in the e- 
portfolio, number of number 
of resumes were uploaded to 
the e-portfolio, and number of 



times resumes were viewed (by 
anyone). Additional information 
about the BGSU e-portfolios can 
be found at http://epsilen.with. 
bgsu.edu 

The first use of e-portfolios 
by students occurred in the 
2003-2004 academic year, as 
they were adopted on a voluntary 
basis by many of the first-year 
student programs. Usage 
was further increased in 2004 
when the University joined the 
National Coalition for Electronic 
Portfolio Research. This 
organization, initially sponsored 
by the American Association for 
Higher Education, is designed to 
facilitate research on the effects 
of e-portfolio participation on 
student learning and success. 
Further details about the 
Coalition can be found at http:// 
ncepr.org/ncepr/drupal/about 

Now in the fourth year of 
implementation, the number of 
e-portfolio accounts has grown 
substantially. Early growth 
came through adoption by a 
few instructors and word-of- 
mouth endorsements among 
students (e.g., "my e-portfolio 
got me an internship!"), along 
with improved stability of the 
software. A major milestone was 
the inclusion of the e-portfolio 
software inside the University's 
portal, enabling a single login 
and navigation among all 
authorized web applications. 
Coincidentally, this user-friendly 
approach is responsible for the 
accounts in which no artifacts 
have been uploaded — a single 
click on an e-portfolio link 
inside the portal will create an 
account, even when the student 
had no intention to do so. A 
comparison of Epsilen with two 
other commercial e-portfolio 
systems in three sections of 
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a sophomore-level Education 
course revealed a preference 
for Epsilen due to its ease of 
use. Epsilen e-portfolios are 
introduced by instructors in some 
first-year experience classes, 
sections of general studies 
writing, and courses in the 
general education program, but 
usage throughout the University 
remains voluntary. 

Pilot Study 

A pilot study (Knight, Hakel, 

& Gromko, 2004) was carried 
out that compared retention 
rates, grade-point averages, and 
credit hours earned between 
the population of 41 BGSU 
undergraduates who had e- 
portfolio artifacts and a matching 
sample of 41 students who 
did not have e-portfolios. The 
comparison was made in this 
way because many students with 
e-portfolio accounts were found 
to have uploaded no artifacts. 
Students with e-portfolio 
artifacts had both significantly 
greater cumulative grade-point 
averages and credit hours earned 
than undergraduates without 
e-portfolio artifacts. There 
was no significant difference 
in retention rates between 
undergraduate students 
with and without e-portfolio 
artifacts. After demographic 
and educational background 
factors were controlled, no 
significant differences were 
found concerning retention or 
grade-point average, although 
significantly greater credit hours 
earned remained for students 
with e-portfolio artifacts. Finally, 
the number of e-portfolio 
artifacts was not significantly 
related to retention, grade-point 
average, or credit hours earned. 



Research Questions 

This paper describes of a 
second research study with 
a much larger number of 
participants, which was designed 
to investigate the following 
research questions: 

1 . What are the characteristics of 
students who have e-portfolio 
artifacts and how are such 
students different than others 
at BGSU? 

2. What significant differences 
exist in retention rates, 
grade-point averages, and 
credit hours earned for BGSU 
students who have e-portfolio 
artifacts, those who have 
e-portfolio accounts but no 
artifacts, and a control group 
of students who did not create 
e-portfolios? 

3. What significant differences 
exist in students' self-reported 
academic engagement for 
BGSU students who have e- 
portfolio artifacts, those who 
have e-portfolio accounts but 
no artifacts, and a control 
group of students who did not 
create e-portfolios? 

4. Are there significant 
relationships between various 
artifact measures (number of 
showcase artifacts, number 
of matrix artifacts, number of 
artifact-specific reflections, 
number of general reflections, 
total number of files uploaded 
to the e-portfolio, number of 
events posted to the student's 
e-portfolio calendar, number 
of bookmarks created in the 
e-portfolio, number of number 
of resumes uploaded to the e- 
portfolio, and number of times 
the resumes were viewed) and 
retention rates, cumulative 
grade-point averages, and 
cumulative credit hours 



earned for students who have 
e-portfolio artifacts? 

5. Does having e-portfolio 
artifacts significantly 
predict retention, grade- 
point average, and credit 
hours earned after student 
background factors (gender, 
race, age, high school 
grade-point average, living 
arrangements, and college) 
are controlled? 

Method 

Data from all 2004-2005 
undergraduate e-portfolio 
accounts were extracted from 
the e-portfolio database in July 
2005. Students included both 
those at BGSU's main campus 
and at Firelands, its associate- 
degree-granting regional college 
located in Huron, Ohio. While 
1,333 accounts existed, an 
inspection of the contents of 
each account revealed that 821 
accounts actually contained one 
or more artifacts. The number of 
showcase artifacts (artifacts in the 
showcase version of the student's 
e-portfolio), matrix artifacts 
(artifacts in the matrix version of 
the student's e-portfolio), artifact- 
specific reflections, general 
reflections, total files uploaded 
to the e-portfolio, events posted 
to the student's e-portfolio 
calendar, bookmarks created 
in the e-portfolio, number of 
number of resumes uploaded 
to the e-portfolio, and number 
of times the resumes were 
viewed were recorded for each 
e-portfolio. Demographic and 
educational outcome data were 
collected for (a) the students 
with e-portfolio artifacts (n = 

821), (b) the students who had 
created e-portfolio accounts 
but had no artifacts in their e- 
portfolios (n = 512), and (c) a 
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random sample of 821 students 
who had no e-portfolio accounts 
(control group). Demographic 
data consisted of sex, race, age, 
college, class rank, academic 
status (good standing. Dean's 
List, probation, suspension), 
living arrangements (on- or off- 
campus), high school grade-point 
average, and ACT composite 
score. Educational outcome 
data included retention from 
the Spring 2004 to Fall 2005 
semesters, cumulative grade- 
point average, and student credit 
hours earned as of the conclusion 
of the Spring 2005 semester. 

Undergraduate students with 
e-portfolios were similar to all 
BGSU undergraduate students 
except for their distribution by 
college, class rank, and gender. 
Therefore undergraduate 
students in the control group 
were proportionately matched 
by college, class rank, and 
gender to undergraduate 
students in the e-portfolio 
groups. Scale scores from the 
Fall 2004 administration of the 
BGSU New Student Transition 
Questionnaire (NSTQ), one 
indicator of student academic 
engagement) administered to 
new freshmen, were also included 
in the database. NSTQ scores 
were available for 239 (35%) of 
the freshmen in the-e-portfolio 
groups and 151 (35%) of the 
control group freshmen. Data 
from the National Survey of 
Student Engagement (NSSE) 
were available for 106 (16%) 
of the freshmen in the e- 
portfolio groups and 46 (11%) 
of the control group freshmen. 
Descriptive, univariate, and 
multivariate statistical analyses 
were used to address the 
remaining research questions. 



Results 

Table 1 describes and compares 
the population of BGSU students 
with e-portfolio artifacts, with 
e-portfolio accounts but without 
artifacts, and the control group 
without e-portfolio accounts. 
Students with e-portfolio artifacts 
were significantly more likely to 
be female (%^ = 15.7, df = 2, p 

< .001), students of color (%^ = 
44.9, df= 14, p < .001), in the 
College of Musical Arts = 

73.3, cff= 16, p<. 001), on the 
Dean's List (f = 167.5, df= 15, 
p < .001), and live on campus 

= 128.5, df=2,p< .001). Those 
with e-portfolio artifacts were also 
likely to have higher ACT scores 
(F=4.2, cff= 2, 1840, p<. 05), 
higher high school grade-point 
averages (F= 19.2, df= 2,1889, p 

< .001), higher cumulative college 
grade-point averages (F = 21 .7, 
df= 2,1986, p < .001), and higher 
credit hours earned (F = 21 .7, df 
= 2,1986, p < .001). Those in the 
control group were significantly 
likely to be older students (F = 
94.7, df= 2,2151, p< .001). 

Retention rates to Fall 2005 
were significantly different 
between groups. Those with 
e-portfolio artifacts had higher 
retention rates than those with e- 
portfolio accounts but no artifacts, 
who, in turn, had higher retention 
rates that those in the control 
group (see Table 2). Please note 
that the sum of students retained 
and not retained does not equal 
the total number of students 
by group because 236 of the 
students graduated in May or 
August 2005. 

As noted in Table 3, 
undergraduates who had e- 
portfolio artifacts showed 
significantly higher grade-point 
averages than either those with 
e-portfolio accounts but no 



artifacts or the control group. 
Also, students with e-portfolio 
artifacts had significantly greater 
credit hours earned as compared 
with the control group. Finally, 
undergraduates with e-portfolio 
accounts but with no artifacts 
had significantly greater credit 
hours earned than students in the 
control group. 

Tables 4 and 5 indicate that 
there were no statistically 
significant group differences in 
any of the NSTQ or NSSE scales, 
which serve as measures of 
student academic engagement. 
See Kuh (2001) for details about 
the NSSE items and scales 
and see http://www.bgsu.edu/ 
offices/ir/studies/transition/ 
newstudent06.htm for a listing of 
the NSSE items. 

Since the majority of 
undergraduate students with 
some e-portfolio artifacts were 
missing showcase artifacts, 
artifact-specific and general 
reflections, and events posted 
to calendars, bookmarks, and 
resumes, only the number of 
matrix artifacts and total number 
of files uploaded were related to 
retention rates through logistic 
regression analysis. As shown 
in Table 6, neither of these 
relationships was statistically 
significant. It is likely that many 
students are missing several of 
the e-portfolio elements because 
their use remains voluntary and 
few classes beyond those in the 
first year include assignments 
where students are asked to use 
e-portfolios. 

Table 7 highlights significant 
positive correlations between 
grade-point average and number 
of showcase artifacts, total 
number of files uploaded, and 
number of resumes uploaded, 
and between credit hours 
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Table 1 

Descriptive Statistics for 2004—2005 BGSU Students by Group 






Characteristic 


With e-portfolio 
Artifacts 

N % 


Group 

Without e-portfolio 
Artifacts 


Control Group 


Race/Ethnicity 


American Indian 


2 


0.2% 


5 


1.0% 


3 


0.4% 


Asian 


2 


0.2% 


3 


0.6% 


7 


0.6% 


Black 


85 


10.4% 


46 


9.0% 


44 


5.4% 


Hispanic-Latino 


14 


1.7% 


13 


2.5% 


34 


3.0% 


Hispanic-Other 


0 


0.0% 


0 


0.0% 


4 


0.0% 


White 


689 


83.9% 


434 


84.8% 


708 


86.2% 


Other 


5 


0.6% 


4 


0.8% 


1 


0.1% 


Unknown 


24 


2.9% 


7 


1.4% 


20 


2.4% 


Sex 


Female 


594 


72.4% 


319 


62.3% 


577 


70.3% 


Male 


227 


27.6% 


227 


37.7% 


244 


29.7% 


College 


Arts and Sciences 


169 


20.6% 


130 


25.4% 


169 


20.6% 


Business Administration 


96 


11.7% 


45 


8.8% 


97 


11.8% 


Education and Human Development 


297 


36.2% 


144 


28.1% 


299 


36.4% 


Firelands 


6 


0.7% 


0 


0.0% 


6 


0.7% 


Health and Human Services 


59 


7.2% 


53 


10.4% 


61 


7.4% 


Musical Arts 


61 


7.4% 


5 


1.0% 


56 


6.8% 


Technology 


27 


3.3% 


26 


5.1% 


27 


3.3% 


Undeclared 


101 


12.3% 


107 


20.9% 


101 


12.3% 


Non-Degree 


5 


0.6% 


2 


0.4% 


5 


. 0.6% 


Class Rank 


Freshman 


429 


52.3% 


255 


49.8% 


426 


51.9% 


Sophomore 


189 


23.0% 


111 


21.7% 


189 


23.0% 


Junior 


101 


12.3% 


71 


13.9% 


108 


13.0% 


Senior 


97 


11.8% 


73 


14.3% 


93 


11.3% 


Non-Degree 


5 


0.6% 


2 


0.4% 


5 


0.6% 


Academic Status 


Dean's List 


256 


31.2% 


125 


24.4% 


123 


15.0% 


Good Standing 


508 


61.9% 


322 


62.9% 


506 


61.6% 


Warning 


36 


4.4% 


31 


6.1% 


77 


9.4% 


Probation 


4 


0.5% 


15 


2.9% 


60 


7.3% 


Suspension 


12 


1.5% 


15 


2.7% 


52 


6.3% 


Not Applicable 


5 


0.6% 


4 


0.8% 


1 


0.1% 


Living Arrangements 


On-Campus 


544 


66.3% 


337 


65.8% 


338 


41.2% 


Off-Campus 


277 


33.7% 


175 


34.25 


483 


58.8% 



earned and total number of 
files uploaded and number of 
resumes uploaded. It may be 
that students with a greater 
number of credit hours are closer 
to graduation and are posting 



resumes to their e-portfolios as 
they seek employment. 

Table 8 shows that 
undergraduates with e-portfolios 
were better retained after sex, 
race, age, high school grade- 



point average, college, and living 
arrangements were controlled. 
Having e-portfolio artifacts had 
significantly positive effects 
upon grade-point average (see 
Table 9) and credit hours earned 
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Table 1 (continued) 

Descriptive Statistics for 2004—2005 BGSU Students by Group 


With e-portfolio 
Artifacts 

Characteristic M 


SD 


Group 

Without e-portfolio 
Artifacts 
M SD 


Control Group 
M SD 


High School Grade-Point Average 


3.31 


0.51 


3.18 


0.52 


3.14 


0.55 


ACT Score 


21,95 


4.05 


21.30 


3.61 


21.79 


3.76 


Age 


20.08 


2.57 


20.03 


2.49 


22.91 


6.83 


Number of Times the Resume Page Was Viewed 


6.46 


12.07 










Number of Bookmarks 


2.27 


1.84 










Number of Events Posted to Calendar 


5.37 


11.37 










Number of Files Uploaded 


5.16 


7.36 










Number of Showcase Artifacts 


2.73 


2.48 










Number of Matrix Artifacts 


4.47 


6.49 










Number of Reflections Associated with Documents 


4.84 


3.59 










Number of General Reflections 


4.51 


3.72 










Number of Resumes Uploaded 


1.01 


0.12 










Cumulative Grade-Point Average 


3.03 


0.68 


2.85 


0.79 


2.79 


0.80 


Cumulative Credit Hours Earned 


62.95 


37.95 


61.87 


40.44 


55.65 


40.38 


NSTQ Social Adjustment Scale 


29.18 


4.55 


29.12 


4.55 


29.40 


4.56 


NSTQ Academic Adjustment Scale 


10.29 


3.18 


10.43 


3.26 


9.70 


2.93 


NSTQ Satisfaction with Living Arrangements Scale 


11.13 


2.50 


11.67 


2.40 


11.23 


2.59 


NSTQ University Involvement Scale 


8.88 


2.60 


9.34 


1.96 


9.30 


2.27 


NSTQ Qther Involvement Scale 


7.95 


1.38 


7.93 


1.40 


7.62 


1.50 


NSSE Academic Challenge 


53.98 


14.27 


60,29 


12.53 


55.29 


11.50 


NSSE Active and Collaborative Learning 


45.89 


16.52 


49.02 


15.99 


46.48 


17.06 


NSSE Student-Faculty Interaction 


42.92 


18.68 


51.31 


18.82 


48.00 


21.93 


NSSE Enriching Educational Experiences 


30.02 


13.78 


31.90 


15.42 


31.20 


12.79 


NSSE Supportive Campus Environment 


60.67 


16.74 


69.26 


13.70 


62.69 


14.28 



Table 2 

Difference in Retention Rates to Fall 2005 by Group 



Retained Not Retained (1) 



Students with e-portfolio Artifacts 


685 


82 


(89.3%) 


(10.7%) 


Students with e-portfolio Accounts 
but No Artifacts 


361 


75 




(82.8%) 


(17.2%) 


Control Group 


515 


200 




(72.0%) 


(28.0%) 



Note. *** p < .001. 
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Table 3 



Mean Differences in Spring 2005 Cumulative Grade-Point Average and Credit 
Hours Earned hy Group 





M 


SD 


F(2, 1986) 


Grade-Point Average 


Students with e-portfolio Artifacts 


3.03= 


0.68 


21 70*** 


Students With e-portfolio Accounts but No Artifacts 


2.85 


0.79 




Control Group 


2.78 


0.80 




Credit Hours Earned 


Students with e-portfolio Artifacts 


63.0^- 


37.9 


6.85** 


Students With e-portfolio Accounts but No Artifacts 


61.9= 


40.4 




Control Group 


55.7 


40.4 





Note. g_< .01 . p< .001 . a = The group of students with e-portfolio artifacts was significantly different from the other two 

groups at p < .001 (d =.06, d =.33). b = The group of students with e-portfolio artifacts was significantly different from the control 
group at p < .01 (d =.18). c = The group of students with e-portfolio accounts but with no artifacts was significantly different from 
the control group at p < .05 (d =.03). d = effect size or (M^-M 2 /SD^), see Cohen, 1988. 



Table 4 

Mean Differences in Fall 2004 New Student Transition Questionnaire (NSTQ) 
Results hy Group for Freshmen 


Group 


M 


SD 


F (2, 387) 


NSTQ Social Adjustment Scale 






0.123 


Students With e-portfolio Artifacts 


29.18 


4.55 




Students With e-portfolio Accounts but No Artifacts 


29.12 


5.76 




Control Group 


29.40 


4.56 




NSTQ Academic Adjustment Scale 






2.125 


Students With e-portfolio Artifact 


10.29 


3.18 




Students With e-portfolio Accounts but No Artifacts 


10.43 


3.26 




Control Group 


9.70 


2.93 




NSTQ Satisfaction with Living Arrangements Scale 






1.523 


Students With e-portfolio Artifacts 


11.13 


2.50 




Students With e-portfolio Accounts but No Artifacts 


11.67 


2.40 




Control Group 


11.23 


2.59 




NSTQ University Involvement Scale 






1.657 


Students With e-portfolio Artifacts 


8.88 


2.46 




Students With e-portfolio Accounts but No Artifacts 


9.34 


1.96 




Control Group 


9.30 


2.27 




NSTQ Qther Involvement Scale 






2.328 


Students With e-portfolio Artifacts 


7.95 


1.38 




Students With e-portfolio Accounts but No Artifacts 


7.93 


1.40 




Control Group 


7.62 


1.50 
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Table 5 

Mean Differences in Spring 2005 National Survey of Student Engagement 
(NSSE) Results by Group for Ereshmen 


Group 


M 


SD 


F(2, 151) 


NSSE Academic Challenge Scale 






0.506 


Students With e-portfolio Artifacts 


56.09 


13.61 




Students With e-portfolio Accounts but No Artifacts 


60.11 


11.23 




Control Group 


55.68 


10.02 




NSSE Active and Collaborative Learning Scale 






0.809 


Students With e-portfolio Artifact 


42.46 


13.62 




Students With e-portfolio Accounts but No Artifacts 


44.03 


15.44 




Control Group 


45.96 


17.16 




NSSE Student-Faculty Interaction Scale 






2.611 


Students With e-portfolio Artifacts 


41.95 


17.64 




Students With e-portfolio Accounts but No Artifacts 


50.34 


18.51 




Control Group 


48.41 


22.64 




NSSE Enriching Educational Experiences Scale 






1.817 


Students With e-portfolio Artifacts 


28.92 


11.83 




Students With e-portfolio Accounts but No Artifacts 


25.09 


10.17 




Control Group 


30.86 


12.53 




NSSE Supportive Campus Environment Scale 






1.845 


Students With e-portfolio Artifacts 


62.65 


15.63 




Students With e-portfolio Accounts but No Artifacts 


69.42 


13.88 




Control Group 


63.98 


13.18 





Table 6 

Summary of Logistic Regression Analysis Predicting Retention 

Predictor B SE Wald 



Number of Matrix Artifacts .068 .086 .620 

Total Number of Files Uploaded .002 .097 .001 



Table 7 



Correlations of Spring 2005 Cumulative Grade-Point Average and Credit 
Hours Earned With Artifact Measures 



Artifact Measure 


Grade-Point Average 


Credit Hours Earned 


Number of Showcase Artifacts 


.231* 


.051 


Number of Matrix Artifacts 


.051 


.086 


Number of Artifact-Specific Reflections 


.043 


.190 


Number of General Reflections 


.230 


.157 


Total Number of Files Uploaded 


.360*** 


.093* 


Number of Events Posted to Calendar 


.003 


.021 


Number of Bookmarks 


-.020 


.134 


Number of Resumes Uploaded 


.110* 


.287*** 


Number of Times the Resume Page Was Viewed 


-.134 


-.134 



Note. g_< .01 . p< .001 . 
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(see Table 10) after background 
factors were controlled. 

Discussion 

The population of students with 
e-portfolios, while considerably 



larger than the one used for 
the earlier pilot study, still 
represents a relatively small 
proportion of all students at 
the University and is skewed in 
terms of several demographic 
and educational factors. More 



importantly, students' utilization 
of e-portfolios at BGSU remains 
a voluntary activity, and there is 
no way to control for differences 
in motivation between students 
with e-portfolios and others as 
comparisons are made. 



Table 8 



Summary of Logistic Regression Analysis Predicting Retention After 
Controlling for Gender, Race, Age, High School Grade-Point Average, College, 
and Living Arrangements 



Predictor 


B 


SE 


Wald 


Gender (Female) 


.324 


.148 


4.812* 


Race (Student of Color) 


-.099 


.204 


.237 


Age 


.010 


.021 


.225 


High School Grade-Point Average 


.902 


.145 


38.795*** 


College: Arts and Sciences 


-.328 


.187 


3.075 


College: Education and Human Development 


.110 


.167 


.437 


College: Musical Arts 


.187 


.437 


.183 


College: Technology 


.378 


.397 


.904 


Living Arrangements (On-Campus) 


.761 


.153 


24.570*** 


e-portfolio Group (With e-portfolio Artifacts) 


.730 


.141 


26.925*** 



Note. g_< .01 . p< .001 . Change in Cox & Snell after entry of e-portfolio group = .02. 



Table 9 

Summary of Regression Analysis Predicting Spring 2005 Cumulative Grade- 
Point Average After Controlling for Gender, Race, Age, High School Grade- 
Point Average, ACT Score, College, and Living Arrangements 


Predictor 


B 


SE 


Wald 


Gender (Female) 


0.078 


.034 


.048^^ 


Race (Student of Color) 


-0.250 


.047 


- 110*** 


Age 


0.032 


.007 


104*** 


ACT Score 


0.026 


.005 


1 32*** 


High School Grade-Point Average 


0.678 


.037 


.465*** 


College: Arts and Sciences 


0.030 


.051 


.016 


College: Business Administration 


-0.021 


.059 


-.009 


College: Education and Human Development 


0.119 


.046 


.077* 


College: Firelands 


0.177 


.229 


.016 


College: Health and Human Services 


0.108 


.068 


.037 


College: Musical Arts 


0.143 


.078 


.044 


College: Technology 


0.144 


.089 


.035 


Living Arrangements (On Campus) 


0.076 


.036 


.048* 


e-portfolio Group (With e-portfolio Artifacts) 


0.160 


.030 


.106*** 


Note. = .38 (df = 1602, p < .001). Change in 

*p<.05. ***p<.001. 


R^ after entry of e- 


-portfolio group = .02. 
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Table 10 

Summary of Regression Analysis Predicting Spring 2005 Cumulative Credit 
Hours Earned After Controlling for Gender, Race, Age, High School Grade- 
Point Average, ACT Score, College, and Living Arrangements 


Predictor 


B 


SE 


Wald 


Gender (Female) 


1.093 


1.532 


.014 


Race (Student of Color) 


1.423 


2.069 


.013 


Age 


5.621 


0.318 


.374*** 


ACT Score 


0.332 


0.219 


.034 


High School Grade-Point Average 


6.736 


1.658 


.093*** 


College: Arts and Sciences 


8.785 


2.252 




College: Business Administration 


15.111 


2.614 


'1 27*** 


College: Education and Human Development 


8.376 


2.045 


109*** 


College: Firelands 


-21.177 


10.165 


-.039^^ 


College: Health and Human Services 


12.907 


3.033 




College: Musical Arts 


52.935 


3.481 


.325*** 


College: Technology 


11.888 


3.934 


.058** 


Living Arrangements (On Campus) 


-25.106 


1.670 


-.320*** 


e-portfolio Group (With e-portfolio Artifacts) 


7.221 


1.348 


.096*** 


Note. R2 = .51 (df = 1602, p < .001). Change 
*p<.05. ***p<.001. 


in after entry of e- 


-portfolio group = .03. 





Despite these limitations, the 
current study provides intriguing 
evidence about the relationship 
between student success and the 
use of e-portfolios. Its results 
suggest that e-portfolios may 
indeed serve as a key tool for 
providing meaningful, authentic 
feedback to improve student 
learning. More widespread 
use of e-portfolios might help 
institutions to deal with the 
problems noted at the beginning 
of this paper. 

At BGSU and elsewhere, 
though, e-portfolios must not 
only be used widely among 
students, but all of their features 
must be exploited. Another 
major milestone in BGSU's 
implementation of this tool has 
not yet occurred (although it is in 
development). We are currently 
developing rubrics for learning 
outcomes and student reflections 
that, when implemented, will 



provide faculty and advisors 
in various disciplines a reliable 
measurement of student 
learning as documented in e- 
portfolios. The next phase of our 
research efforts will follow this 
implementation. Future studies 
will also use logistic regression 
to examine why some students' 
e-portfolio accounts include 
artifacts while other students' do 
not. 

Siemens (2004) lists the 
conditions necessary for e- 
portfolios to be successfully 
implemented: 

• The e-portfolio is viewed as 
a personal, learner-in-control 
tool. It is treated as central to 
the learning and assessment 
process. 

• Learners are introduced to the 
concept and instructed on how 
to use the system (both from 

a technical perspective and 



from a "how will this help you" 
perspective). 

• The curriculum has been 
designed to require learners 
to use the e-portfolio in 
completing their course work 
and assignments. 

• The e-portfolio is used for 
assessment of learning objec- 
tives. Instructor feedback can 
be integrated back into the 
e-portfolio and treated as an 
artifact. 

• Learners are provided 
staged advising sessions 
evaluating their effective use 
of e-portfolios (this is a meta- 
cognitive evaluation of e- 
portfolio use). 

• An e-portfolio culture 
(Gathercoal, Love, Bryde, 

& McKean, 2002) exists, 
encouraging learners to include 
personal life experiences. 
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awards, non-academic 
activities, and other character/ 
learning revealing artifacts in 
their e-portfolios. 

• Dialogue, debate, discussion, 
and examples of e-portfolio use 
are common. 

• Time is allotted for e-portfolio 
development. 

• Faculty understand and 
promote the value of e- 
portfolios. 

• Technical details are well 
managed, resulting in a simple, 
positive end-user experience. 

At BGSU and across most 

colleges and universities, we are 
at the early stage of creating such 
conditions. Time will tell whether 
e-portfolios will fully realize their 
potential to improve student 
learning. 
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